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Abstract 

It is pointed out that the case for Shannon entropy and von Neu- 
mann entropy, as measures of uncertainty in quantum mechanics, is not 



as bleak as suggested in quant-ph/0006087. The main argument of the 



latter is based on one particular interpretation of Shannon's H-function 
(related to consecutive measurements), and is shown explicitly to fail 
for other physical interpretations. Further, it is shown that Shannon 
and von Neumann entropies have in fact a common fundamental sig- 
nificance, connected to the existence of a unique geometric measure of 
uncertainty for classical and quantum ensembles. Some new properties 
of the "total information measure" proposed in :[uant-ph/0006087| are 
also given. 

I Interpretations of Shannon entropy 

Note that the term "Shannon entropy" will be used throughout for Shan- 
non's H function, rather than the term "Shannon information" used in [1], 
as the latter quantity in general involves a difference of entropies [|[ . 

The main argument given in [1] , against the use of Shannon entropy for 
quantum measurements, relies on showing that a particular interpretation 
of this quantity (involving consecutive measurements) does not accord with 
quantum mechanics. Here it is pointed out that an alternative interpretation 
may be given which does not involve consecutive measurements in any way. 
Hence no general conclusions on the adequacy of Shannon entropy can be 
drawn from the argument in [|^. 

Shannon showed that his H function could be derived from a set of 
axioms for "uncertainty" or "randomness" Numerous minor variations 
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on these axioms have since been used |Q, and the discussion in Sec. Ill 
of [1] centres on the Faddeev form of the so-called "grouping axiom" ^] , 

H{pi,p2, ...,Pn-l,qi,q2) = H{pi,p2, ...,Pn) +PnH{qi/pn,q2/Pn), (1) 

for discrete probability distributions {pi,...,pn) and {qi/pn,q2/Pn) (where 
Pn = qi + q2)- This axiom, together with axioms for the continuity and 
symmetry of H with respect to its arguments, leads uniquely to 

H{pi,p2,...,Pn) = -C^Pilnpi, (2) 

i 

where C is an arbitrary multiplicative constant 

Now, Brukner and Zeilinger induce their interpretation of Shannon en- 
tropy, in Sec. Ill of [1], via an interpretation of the axioms from which it is 
derived. Since the Faddeev form (|^) of the grouping axiom is not physically 
transparent, they introduce a physical justification for it based on consecu- 
tive measurements and their joint probabilities. They then reject this axiom 
(and consequently the Shannon entropy) as "inadequate" for quantum mea- 
surements, essentially because classical joint probabilities do not exist for 
noncommuting quantum observables. 

However, the above argument immediately becomes inapplicable when 
an alternative form of the grouping axiom, with a physical justification in 
which consecutive measurements play no part, is used. In particular, define 
two distributions to be non- overlapping if and only if there is some mea- 
surement which can distinguish between them with certainty. Thus if some 
outcome has a non-zero probability of occurrence for one such distribution, 
then it has a zero probability of occurrence for the other. 

Suppose now that one prepares a mixture of non-overlapping distribu- 
tions, each having its own "randomness" or "uncertainty". The randomness 
of outcomes is then expected to increase on average, due to the information 
thrown away by mixing, ie, due to the randomness arising from the mixing 
probabilities. The grouping axiom may then be formulated as requiring this 
expected increase to be additive |5|: 

the randomness of a mixture of non- overlapping distributions is equal to 
the average randomness of the individual distributions, plus the randomness 
of the mixing distribution. 

For example, note that the (1/2, 1/3, 1/6) distribution discussed in Sec. 
Ill of [1] is equivalent to an equally weighted mixture of the two non- 
overlapping ensembles (1,0,0) and (0,2/3,1/3). The above form of the 
grouping axiom then implies that 

H{l/2, 1/3, 1/6) = ^H{1, 0, 0) + ^H{0, 2/3, 1/3) + H{l/2, 1/2) (3) 
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(where H{1, 0, 0) is easily shown to vanish - see below). Further, the Faddeev 
form of the grouping axiom in Eq. (|l|) is recovered by decomposing the distri- 
bution {pi,p2, -tPu-i, qi, q2) into the mixture of non-overlapping ensembles 
(1,0,. .,0,0,0), (0,1,. ..,0,0,0), (0,0, ...,1,0,0), (0,0,...,0,gi/p„,g2/Pn), 
with respective mixing coefficients pi,P2, Pn, and noting that H{1, 0, 0, 0, 0) 
etc. must vanish (this vanishing of uncertainty is of course expected for such 
distributions, and follows immediately when the above form of the group- 
ing axiom is combined with the symmetry axiom for H, for the case of an 
equally weighted mixture of such distributions). 

The grouping axiom thus has an alternative form with a physical inter- 
pretation involving only the notions of mixtures and non-overlapping ensem- 
bles, and having no reference to consecutive measurements. It follows that 
the argument in Sec. Ill of [1] shows only that the particular interpretation 
of the Shannon entropy given there, rather than the Shannon entropy itself, 
is "inadequate". 

II Geometric and operational significance of Shan- 
non and von Neumann entropies 

In Sec. IV of [1], Brukner and Zeilinger make the valid point that the 
Shannon entropy of a quantum measurement is not invariant under unitary 
transformations, and hence is not suitable as an invariant measure of "un- 
certainty" or "randomness" for quantum systems. They further note that, 
for example, simply summing up Shannon entropies for three orthogonal 
spin directions of a spin-1/2 particle does not provide an invariant measure. 
However, they reject the von Neumann entropy as an appropriate generali- 
sation for the uncertainty of a quantum system, essentially on the grounds 
that it is equal to a Shannon entropy only for the "classical" case of a mea- 
surement diagonal in the same basis as the density operator of the system 
(although the same "criticism" holds for their proposed measures I{p) and 
I{p) in Eqs. (|) and (0) below). 

I wish to point out that there is in fact a very deep connection between 
Shannon and von Neumann entropies as measures of uncertainty, connected 
to the existence of a unique measure of uncertainty for classical and quantum 
systems with the geometric properties of a "volume". In particular, consider 
a measure of the volume (or spread) of a classical or quantum ensemble, 
which satisfies: 

(i) the volume of any mixture of non- overlapping ensembles, each of equal 
volume, is no greater than the sum of the component volumes ( with equality 
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for an equally-weighted mixture) 

(ii) the volume of an ensemble comprising two subsystems is no greater 
than the product of the volumes of the subsystems (with equality when the 
subsystems are uncorrelated) 

(iii) the volume of an ensemble is invariant under all measure-preserving 
transformations of the underlying space. 

These postulates are seen to be independent of whether the ensemble is 
classical or quantum, and are discussed in detail in (where the second 
postulate is shown to correspond to the Euclidean property that the product 
of the lengths obtained by projecting a volume onto orthogonal axes is never 
less than the original volume). It is shown in that the only continuous 
measure of uncertainty V which satisfies these postulates is 

V = Ke^, (4) 

where K is a multiplicative constant, and S denotes the Shannon entropy for 
classical ensembles and the von Neumann entropy for quantum ensembles. It 
is worth noting that this result also holds for the case of continuous classical 
distributions. 

Thus the exponential of the entropy is a fundamental geometric measure 
of uncertainty for both quantum and classical ensembles. Indeed, volume 
may be taken as the primary physical quantity, and entropy then defined 
as its logarithm. Note that this approach to entropy is very different to 
the axioms discussed in Sec. I above, and leads to an additive rather than 
a multiplicative constant for entropy. Further discussion and applications 
may be found in 

It is concluded that, in the context of uncertainty measures, the von Neu- 
mann entropy is in fact an appropriate quantum generalisation of Shannon 
entropy. 

It is also perhaps worth making some remarks on the "operational" sig- 
nificance of von Neumann entropy, in the light of the discussion in [1] . Sup- 
pose that one makes sufficient measurements on copies of a quantum ensem- 
ble to be able to accurately estimate the density operator of the ensemble. 
For example, as noted in [1], this may be done if the distributions of a 
sufficient number of non-commuting observables are accurately determined 
(and is of course the basis of quantum tomography). It follows that, having 
the density operator at hand, one can immediately calculate the von Neu- 
mann entropy. The latter quantity thus has a perfectly good "operational" 
definition, and indeed differs in this regard no more and no less from any 
functional of the density operator, including the "total information" pro- 
posed in Sec. V of [1] (see also Eq. below). 
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Ill Some properties of "total information" 



Brukner and Zeilinger propose a measure of "total information" for quantum 
systems which has some interesting properties. Here I would like to point 
out some further relationships, and also to demonstrate a property similar 
to the additivity property found in [1], but which does not depend on the 
existence of a complete set of mutually complementary observables (which 
at present are only known to exist for Hilbert space dimensions that are 
prime or powers of 2). 
First let 

denote the information measure defined in Eq. (17) of [1] for distribution 
{pi, ...,Pn)- If this distribution is generated by measurement of some Her- 
mitian observable A on an n-dimensional Hilbert space, then it may alter- 
natively be denoted by I (A). Now, suppose that the Hilbert space admits a 
complete set of mutually complementary observables, i.e., n + 1 observables 
Ai, ...,An^i such that the distribution of any one observable is uniform for 
an eigenstate of any other j^. It follows that one has the general "recon- 
struction" property |^] 

P = J2p{A,)-1, (6) 

i 

where p{Ai) denotes the density operator corresponding to a projective mea- 
surement of Ai on an ensemble described by p. As shown in Sec. V of [1], 
the additivity property 

I{A,) = tr[(p - l/n)2] = tT[p'] - 1/n =: /(p) (7) 

i 

then follows, where I{p) is the natural quantum generalisation of I{p), called 
the "total information" [1]. Thus the quantum information measure is just 
the sum of the classical information measures, over a complete set of mutu- 
ally complementary observables. 

Eq. (0) is a very nice property relating the quantum and classical con- 
texts. Noting that the second term in each of Eqs. (||) and (|^) can be 
interpreted as the square of the "distance" between the state of the system 
and a maximally-random state, this additivity property can be viewed as a 
type of Pythagorean connection between quantum and classical distances. 
It may also be re-expressed as a relation between the quantum and classical 



5 



"inverse participation ratios" |||, |^ 

R{p) = [iTp']-\ R{p) = [Y^p]Y^ (8) 

3 

(corresponding to non-Euclidean measures of "volume" In particular, 

Eq. (|^) implies that 

l/R{p)=Y,l/R{A,)-l, (9) 

i 

which is formally analogous to the reconstruction property Eq. (^). 

However, as noted in [1], the existence of n-|- 1 mutually complementary 
observables has in fact only been shown for the cases that n is prime or a 
power of 2. This puts the general applicability of Eq. (|7|) in doubt. 

I wish to point out here that there is in fact a similar relation between 
I{p) and I{p) which does not depend on the existence of a complete set of 
complementary observables. In particular, instead of summing I{A) over a 
specific group of observables, one can instead average I{A) over all (non- 
degenerate Hermitian) observables. Such observables differ only by unitary 
transformations, and if dU denotes the normalised invariant Haar measure 
over the group of unitary transformations {f/}, it can be shown (see Ap- 
pendix) that 

I{p) = (n + 1) j I{UAU^)dU. (10) 

Thus the quantum information measure is proportional to the average of 
the classical measure, over all observables. This is clearly similar in spirit 
to Eq. (1^, which corresponds to replacing the average over all observables 
in Eq. ([lO|) by an average over n + 1 mutually complementary observables. 
The two averages are thus equivalent for this information measure. 
APPENDIX 

To prove Eq. (0), note that for an observable A with eigenstates {|aj)} 
that 

I{A) = Y,{a,W3?-'^/n. (11) 

3 

For each term in the sum, the average over all observables is independent 
of \aj) (since it is unitarily invariant), and hence \aj) may be replaced by a 
common state, \a) say, to give 

j I{UAU^)dU = n j {a\p\afdVLa - 1/n, (12) 
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where dQa is the normahsed invariant measure over pure states By 
expanding p into eigenstates, and again noting the unitary invariance of the 
average, it follows that the righthand side of this equation has the form 
atr[p^] + /3, where a and f3 are definite integrals. Further, noting that the 
righthand side vanishes for the maximally mixed state p = (l/n)l, one must 
have P = —a/n. Hence 

j I{UAU^)dU = al{p). (13) 

Finally, to determine q, assume that p corresponds to some pure state 
|6). Hence I{p) = (n— l)/n from Eq. (0), and substitution in Eqs. (|T^) and 
(ID yields 



a 



/(n-1). (14) 



The integral may be evaluated either via Eq. (38) of (which gives the 
probability distribution for the variable Y = | (a|6) p), or via the more general 
method in Appendix A of |10|, as 2/n/(n + 1), and Eq. (pX|) immediately 



follows. 
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